Information processing apparatus, information processing system, and information processing method

ABSTRACT

There is provided an information processing apparatus including a waveform generation unit configured to generate a waveform according to a relative movement between two objects in a virtual space, and an output control unit configured to control a haptic output and a sound output on the basis of frequency distribution of the waveform generated by the waveform generation unit.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2014-045056 filed Mar. 7, 2014, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND

The present disclosure relates to an information processing apparatus,an information processing system, an information processing method, anda program.

In order to increase a sense of reality of content such as a movie or avideo game, a technique to give users a haptic output such as vibrationsaccording to moving images or sounds has been commonly used in recentyears. Accordingly, products related to such a technique have beenbecoming commercially available.

For example, JP 2009-72600A discloses a technique related to anapparatus that monitors audio signals, video signals, data signals, andthe like outputted from software, and when these signals satisfypredetermined conditions, outputs sensory control signals to a vibrationsensory actuator, for example.

SUMMARY

However, in general, it becomes difficult for a haptic output actuatorto perform an output in response to all input signals when the frequencyof an input signal is higher than a range within which an output ispossible. Accordingly, some input signals are not outputted due to thefrequency, and a user may experience a feeling of strangeness.

Note that it is known that, when frequencies of haptic vibrations becomehigher, human perception of the vibrations is changed from hapticperception to auditory perception.

Accordingly, an embodiment of the present disclosure provides aninformation processing apparatus, an information processing system, aninformation processing method, and a program which can achieve a morerealistic sensory output.

According to an embodiment of the present disclosure, there is providedan information processing apparatus including a waveform generation unitconfigured to generate a waveform according to a relative movementbetween two objects in a virtual space, and an output control unitconfigured to control a haptic output and a sound output on the basis offrequency distribution of the waveform generated by the waveformgeneration unit.

According to another embodiment of the present disclosure, there isprovided an information processing system including a waveformgeneration unit configured to generate a waveform according to arelative movement between two objects in a virtual space, and an outputcontrol unit configured to control a haptic output and a sound output onthe basis of frequency distribution of the waveform generated by thewaveform generation unit.

According to another embodiment of the present disclosure, there isprovided an information processing method including generating awaveform according to a relative movement between two objects in avirtual space, and controlling a haptic output and a sound output on thebasis of frequency distribution of the generated waveform.

According to another embodiment of the present disclosure, there isprovided a program for causing a computer to execute a waveformgeneration function of generating a waveform according to relativemovement between two objects in a virtual space, and an output controlfunction of controlling a haptic output and a sound output on the basisof frequency distribution of the waveform generated by the waveformgeneration function.

As described above, according to one or more of embodiments of thepresent disclosure, there are provided an information processingapparatus, an information processing system, an information processingmethod, and a program which can achieve a more sensory haptic output.Note that the effects described above are not limiting. That is, thetechnology according to the present disclosure can exhibit any of theeffects described in the specification or other effects that areapparent from the descriptions in the specification, along with theabove effects or instead of the above effects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an outline of an information processing systemaccording to an embodiment of the present disclosure;

FIG. 2 is a block diagram showing a schematic functional configurationof an information processing system according to a first embodiment ofthe present disclosure;

FIG. 3 shows an example of processing of generating a waveform fromoperation information of an information processing apparatus accordingto the embodiment;

FIG. 4 shows an example of a change in frequency distribution of awaveform generated in an information processing apparatus according tothe embodiment;

FIG. 5 is a sequence diagram conceptually showing processing of aninformation processing system according to the embodiment;

FIG. 6 is a block diagram showing a schematic functional configurationof an information processing system according to a second embodiment ofthe present disclosure;

FIG. 7 shows an example of adjusting processing of signal intensityrelated to a waveform generated from operation information of aninformation processing apparatus according to the embodiment;

FIG. 8 shows another example of a change in frequency distribution of awaveform generated in an information processing apparatus according tothe embodiment;

FIG. 9 is a sequence diagram conceptually showing processing of aninformation processing system according to the embodiment; and

FIG. 10 is an explanatory diagram showing a hardware configuration of aninformation processing apparatus according to an embodiment of thepresent disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Note that the description will be made in the following order.

1. Outline of information processing system according to embodiment ofpresent disclosure

2. First embodiment of present disclosure (example of output based onfrequency distribution of generated waveform)

-   -   2-1. Configuration of information processing system    -   2-2. Processing of information processing system    -   2-3. Modification example

3. Second embodiment of present disclosure (example of output based onoperation pressure)

-   -   3-1. Configuration of information processing system    -   3-2. Processing of information processing system

4. Hardware Configuration

5. Conclusion

1. Outline of Information Processing System According to Embodiment ofPresent Disclosure

First, an outline of an information processing system according to anembodiment of the present disclosure will be described with reference toFIG. 1. FIG. 1 illustrates the outline of the information processingsystem according to an embodiment of the present disclosure.

The information processing system includes an information processingapparatus 100 and a controller 200. The information processing apparatus100 is a stationary apparatus and has a function of performingprocessing for a virtual space, a function of performing processing ofgenerating information for a haptic output on the basis of the result ofthe above processing, and a communication function. The controller 200is a portable apparatus that can be carried and includes an actuatorthat performs the haptic output and a communication function.Accordingly, in the information processing system, the informationprocessing apparatus 100 generates information for the haptic outputbased on the result of processing in the virtual space, the generatedinformation is transmitted to the controller 200, and the controller 200can perform the haptic output on the basis of the received information.

Further, the controller 200 has a function of detecting a user'soperation, and transmits information related to the detected operationto the information processing apparatus 100. Further, the informationprocessing apparatus 100 has a function of detecting the distance from adisplay screen to the user. Accordingly, in the information processingsystem, the information processing apparatus 100 can perform processingin the virtual space on the basis of the information related to theuser's operation and the distance from the display screen to the user,which are detected by the controller 200.

For example, as shown in FIG. 1, the information processing system canbe composed of the information processing apparatus 100 connected to adisplay apparatus 10 and the controller 200 to be operated by a user 1.The information processing apparatus 100 can provide information relatedto the result of processing for the virtual space, such as athree-dimensional moving image, to the display apparatus 10, and thedisplay apparatus 10 can display the provided moving image. Theinformation processing apparatus 100 can also generate haptic outputsignals on the basis of the event and the like that have occurred in thevirtual space, and can transmit the generated haptic output signals tothe controller 200. The controller 200 can operate a haptic outputactuator on the basis of the received haptic output signals.

The controller 200 can also detect an operation by the user 1, such aspressing of a button or displacement of the controller 200, and cantransmit information related to the detected operation to theinformation processing apparatus 100. The information processingapparatus 100 can detect the distance from the display apparatus 10 tothe user 1, and can perform processing for the virtual space on thebasis of the received information and the detected distance.

It should be noted that, in general, it becomes difficult for the hapticoutput actuator to perform an output in response to all input signalswhen the frequency of an input signal is higher than a range withinwhich an output is possible. Accordingly, some input signals are notoutputted due to the frequency, and a user may experience a feeling ofstrangeness. Further, it is known that, when frequencies of vibrationsthat have been first recognized by haptic perception become higher to bein an audible area, humans recognize the vibrations by auditoryperception. For example, first, when a person moves a finger while thefinger is in contact with a floor, vibrations are generated and she orhe senses the vibrations by haptic perception. Next, when she or hemoves the finger faster, frequencies of vibrations become higher.Accordingly, fewer vibrations are sensed by haptic perception, and,vibrations begin to be sensed as sounds instead, by auditory perception.Further, when the pressure of the finger becomes stronger, vibrationsrecognized by haptic perception are increased, and the volume of soundsrecognized by auditory perception is also increased. Accordingly, theinformation processing system according to an embodiment of the presentdisclosure performs a haptic output and a sound output according tofrequency distribution of input signals.

For example, in a case in which the user 1 operates an object that lookslike a hand in a virtual space by using the controller 200 so as to tapa surface of another object that looks like a floor, the informationprocessing apparatus 100 generates a waveform of vibrations on the basisof the tap operation. Next, the information processing apparatus 100generates output signals on the basis of the frequency distribution ofthe generated waveform, and transmits each of the generated signals tothe controller 200. For example, in a case in which the frequencydistribution of the generated waveform includes a frequency in anaudible area, the information processing apparatus 100 generates soundoutput signals in addition to the haptic output signals. Then, havingreceived the haptic output signals and sound output signals, thecontroller 200 performs the haptic output and sound output on the basisof each of the signals.

In this manner, the information processing system according to anembodiment of the present disclosure generates the waveform according tothe movement of the object in the virtual space, and performs the hapticoutput and sound output on the basis of the frequency distribution ofthe generated waveform. Accordingly, an output related to frequenciesthat are not outputted in a case of the haptic output alone becomespossible as a sound output, and the above described phenomenon isreproduced. Thus, it becomes possible for the user to experience a morerealistic output. Note that the information processing apparatus 100 andthe controller 200 according to first and second embodiments arespecified by adding numbers corresponding to the number of embodimentsto the end of reference numerals, such as an information processingapparatus 100-1 and an information processing apparatus 100-2, forconvenience of the description.

2. First Embodiment of Present Disclosure (Example of Output Based onFrequency Distribution of Generated Waveform)

The outline of the information processing system according to anembodiment of the present disclosure has been described above. Next, aninformation processing system according to the first embodiment of thepresent disclosure will be described. In the information processingsystem according to the present embodiment, the information processingapparatus 100-1 generates a waveform according to a relative movementbetween two objects in the virtual space, and generates the hapticoutput signals and sound output signals on the basis of the frequencydistribution of the generated waveform. Further, a controller 200-1performs the haptic output and sound output on the basis of each of thegenerated signals.

2-1. Configuration of Information Processing System

First, a configuration of the information processing system according tothe first embodiment of the present disclosure will be described withreference to FIG. 2. FIG. 2 is a block diagram showing a schematicfunctional configuration of the information processing system accordingto the present embodiment.

The information processing system is composed of the informationprocessing apparatus 100-1 and the controller 200-1 as shown in FIG. 2.

The controller 200-1 includes an operation detection unit 202, acommunication unit 204, a haptic output unit 206, and a sound outputunit 208.

The operation detection unit 202 detects a user's operation with respectto the controller 200-1. Specifically, the operation detection unit 202detects an operation related to an operation unit of the controller200-1 and the state of the controller 200-1, and generates operationinformation indicating content of the detected operation. For example,the operation detection unit 202 can detect pressing of a buttonincluded in the controller 200-1. Further, the operation detection unit202 includes a sensor such as an acceleration sensor or an angularvelocity sensor, and can detect the displacement of the controller200-1.

The communication unit 204 transmits the operation content detected bythe operation detection unit 202 to the information processing apparatus100-1. Specifically, the communication unit 204 transmits the operationinformation indicating the detected operation content to the informationprocessing apparatus 100-1 when the operation detection unit 202 detectsa user's operation. Further, the communication unit 204 receives thehaptic output signals and sound output signals from the informationprocessing apparatus 100-1. For example, the communication unit 204 cancommunicate with the information processing apparatus 100-1 by radiocommunication such as Bluetooth (registered trademark) communication,Wi-Fi (registered trademark) communication, or infrared lightcommunication, or wired communication via a wire or the like.

The haptic output unit 206 performs the haptic output on the basis ofthe haptic output signals. Specifically, when the communication unit 204receives the haptic output signals from the information processingapparatus 100-1, the haptic output unit 206 generates vibrations thatcan be sensed by humans by haptic perception on the basis of thereceived haptic output signals. For example, the haptic output unit 206can be a vibration actuator such as an eccentric motor or a linearresonant actuator (LRA) and can generate vibrations by operating abuilt-in motor, coil, or the like on the basis of a waveform related tothe received haptic output signals.

The sound output unit 208 performs the sound output on the basis of thesound output signals. Specifically, when the communication unit 204receives the sound output signals from the information processingapparatus 100-1, the sound output unit 208 performs the sound output onthe basis of the received sound output signals. For example, the soundoutput unit 208 can be a speaker.

As shown in FIG. 2, the information processing apparatus 100-1 includesa communication unit 102, an operation processing unit 104, a waveformgeneration unit 106, and an output control unit 108.

The communication unit 102 receives the operation information from thecontroller 200-1. The communication unit 102 also transmits the hapticoutput signals and sound output signals to the controller 200-1.

The operation processing unit 104 performs processing for an object inthe virtual space on the basis of the operation information.Specifically, the operation processing unit 104 performs processing ofmoving the object in the virtual space according to the operationcontent indicated by the operation information. The moving processing ofthe object based on the operation information will be described withreference to FIG. 3, for example. FIG. 3 shows an example of processingof generating a waveform from the operation information of theinformation processing apparatus 100-1 according to the presentembodiment.

First, the operation detection unit 202 detects a user's operation withrespect to the controller 200-1, and the communication unit 204transmits the operation information indicating the detected operationcontent to the information processing apparatus 100-1. For example, asshown in the lower left of FIG. 3, when the user moves the controller200-1 from the left to the right in the figure, the operation detectionunit 202 detects the displacement of the controller 200-1, and thecommunication unit 204 transmits information related to the displacementto the information processing apparatus 100-1. Note that part of thevirtual space including the object related to the user's operation canbe displayed by the display apparatus 10, and the user can operate theobject while viewing the object and the like displayed by the displayapparatus 10.

Next, the operation processing unit 104 moves the object serving as atarget of the user's operation in the virtual space on the basis of theoperation information received by the communication unit 102. Forexample, the operation processing unit 104 can calculate informationrelated to the displacement of an object B serving as the target of theuser's operation in the virtual space on the basis of informationrelated to the displacement of the controller 200-1. Then, as shown inthe upper left of FIG. 3, on the basis of the calculated informationrelated to the displacement, the operation processing unit 104 can movethe object B in the virtual space from the left to the right in thefigure in a state in which the object B is in contact with an object A.Note that the operation processing unit 104 may set, as a displacementamount of the object, a displacement amount of the same size as adisplacement amount of the operation information, or a displacementamount obtained by multiplying the displacement amount of the operationinformation by a factor.

In this manner, the object in the virtual space is moved by the user'soperation. Accordingly, since an output is performed on the basis of themovement of the object related to the user's operation, it becomespossible to increase a sense of reality for the output.

Note that an example in which the object is moved in response to theuser's operation has been shown above; however, the object may be movedautomatically. For example, in a case in which the user does not performoperations, an application that controls the virtual space including theobject can move the object. In this case, by the object being movedautomatically, not by the user's operation, so that the informationprocessing system can be applied to an application to which the user'soperation is not performed.

Referring to FIG. 2 again, the configuration of the informationprocessing apparatus 100-1 will be described. The waveform generationunit 106 generates a waveform according to a relative movement betweentwo objects in the virtual space. Specifically, in a case in which twoobjects are moved relatively in a state in which the objects are incontact with each other, the waveform generation unit 106 generates awaveform according to roughness information of a contact face betweenthe objects. For example, the roughness information of the contact facebetween the objects can be the height of a convex and a concave of oneof the objects. More specifically, processing of generating a waveformwill be described in detail with reference to FIG. 3.

First, as described above, the operation processing unit 104 moves theobject serving as the target of the user's operation in the virtualspace. For example, as shown in the upper left of FIG. 3, the object Bin the virtual space can be moved from the left to the right in thefigure in a state in which the object B is in contact with the object A.

When an object touches another object, or when an object is relativelymoved in a state in which the object is in contact with another object,the waveform generation unit 106 acquires the height of a convex and aconcave in the contact face between the two objects. For example, thewaveform generation unit 106 can acquire the height of a convex and aconcave of the object A in the contact face between the object A and theobject B shown in FIG. 3. Note that the height of a convex and a concaveof the object can be managed as attribution information of the object.

Next, the waveform generation unit 106 generates the waveform accordingto the acquired height of a convex for every time when a certain periodof time elapses. For example, the waveform generation unit 106 cantransform the acquired height of a convex into the intensity of a signalto generate the waveforms for every sampling time. For example, thewaveform generation unit 106 can transform the height of a convex to theintensity of the signal by using the intensity of a certain signalcorresponding to the height of a certain convex.

In this manner, the waveform generation unit 106 generates the waveformaccording to the roughness information of the contact face between theobjects. Accordingly, since output is changed according to the roughnessof the surface of the object, the user can recognize the roughness ofthe surface of the object more realistically.

The generated waveform has frequency distribution according to the speedof the relative movement between the objects. Specifically, the positionwhere the height of a convex and a concave is acquired is changed in asampling time, so that the speed of the change of the acquisitionposition of the height of a convex and a concave varies. Accordingly,the generated waveform has different frequencies. For example, a changein the frequency distribution of the waveform according to the speed ofthe relative movement between the objects will be described in detailwith reference to FIG. 4. FIG. 4 shows an example of a change in thefrequency distribution of the waveform generated in the informationprocessing apparatus 100-1 according to the present embodiment.

First, when a relative movement occurs between two objects that are incontact with each other, the waveform generation unit 106 generates awaveform on the basis of the height of a convex and a concave of one ofthe objects. For example, frequencies of the generated waveform can bedistributed as shown in the upper part of FIG. 4. Note that the verticalaxis of the graph shown in FIG. 4 represents the signal intensity andthe horizontal axis thereof represents frequencies.

In the above condition, when only the speed of the relative movementbetween the objects is increased and the same movement is performed onthe objects, the signal intensity of the waveform generated by thewaveform generation unit 106 is not changed, but the speed of the changeof the acquisition position of the height of a convex and a concavebecomes higher per unit time and frequencies of the generated waveformbecome higher. For example, as shown in a lower part of FIG. 4, comparedwith the frequency distribution of the upper part having a lowerrelative speed, the signal intensity is not changed and the frequencydistribution can be shifted to be in a higher range.

In this manner, the waveform generation unit generates a waveform havingfrequency distribution according to the speed of the relative movementbetween the objects. Accordingly, since an output is changed accordingto the speed of the movement between the objects, it becomes possible toincrease the user's sense of reality with respect to the output.

Here, referring to FIG. 2 again, the configuration of the informationprocessing apparatus 100-1 will be described again. The output controlunit 108 controls the haptic output and sound output on the basis of thefrequency distribution of the generated waveform. Specifically, theoutput control unit 108 controls the sound output in a case in which thefrequency distribution of the waveform includes a frequency higher thana certain frequency, and controls the haptic output in a case in whichthe frequency distribution of the waveform includes a frequency lowerthan the certain frequency. For example, the sound output can be anoutput of a waveform related to a frequency that is higher than thecertain frequency in the waveform, and the haptic output can be anoutput of a waveform related to a frequency that is lower than thecertain frequency in the waveform.

Then, the output control unit 108 generates the haptic output signalsand sound output signals which serve as input signals of the abovedescribed haptic output and sound output. Specifically, the outputcontrol unit 108 filters a certain frequency range of the generatedwaveform to generate each of the haptic output signals and sound outputsignals. For example, when a low pass filter (LPF) is used on thegenerated waveform, it is possible to generate the haptic output signalsin which frequencies are attenuated in a frequency range higher than abroken line in the vertical direction shown in the lower part of FIG. 4,for example. Further, when a high pass filter (HPF) is used on thegenerated waveform, it is possible to generate sound output signals inwhich frequencies are attenuated in a frequency range lower than abroken line in the vertical direction shown in the lower part of FIG. 4,for example. Note that the certain frequency can be a frequency near thelower limit of the audible area (hereinafter also referred to as audiblefrequency).

In this manner, the output control unit 108 controls the sound output ina case in which the frequency distribution of the waveform includes afrequency higher than the audible frequency, and controls the hapticoutput in a case in which the frequency distribution of the waveformincludes a frequency lower than the audible frequency. Accordingly,since the sound output and haptic output are controlled on the basis ofthe audible frequency, it becomes possible to achieve a realisticoutput. Further, the output control unit 108 performs control in amanner that the sound output is an output of a waveform related to afrequency that is higher than the certain frequency and the hapticoutput is an output of a waveform related to a frequency that is lowerthan the certain frequency. Accordingly, since the waveform related to afrequency that is suitable for either the sound output or the hapticoutput is outputted by the corresponding output, it becomes possible toperform an output that fits in with the reality.

Further, the output control unit 108 sets, as the input signals of thehaptic output and sound output, the waveform generated by the waveformgeneration unit 106. Accordingly, since an output corresponding to themovement of an object is performed, it becomes possible to give the userthe sense of reality for the output. Furthermore, the output controlunit 108 sets, as the input signals of the haptic output, the waveformfrom which frequencies higher than the certain frequency are filteredfrom the frequency distribution of the generated waveform. The outputcontrol unit also sets, as the input signals of the sound output, thewaveform from which frequencies lower than the certain frequency arefiltered from the frequency distribution of the waveform. Accordingly,since high frequencies that are unlikely to be outputted by the hapticoutput actuator are attenuated, or frequencies that are lower than theaudible frequency are attenuated, it becomes possible to prevent theentrance of noise and the like into the output.

Not that an example of using two kinds of filters of the LPF and the HPFhas been described above; however, in addition to the two kinds offilters, a band pass filter (BPF) may be used, which filters frequenciesin a certain frequency range, and the output control unit 108 may set,as input signals of another haptic output actuator, a waveform subjectedto filtering processing by the BPF. For example, the frequency rangefiltered by the BPF may be a frequency range between the frequency rangefiltered by the LPF and the frequency range filtered by the HPF, whichis a frequency range lower than the certain frequency, i.e., the audiblefrequency. Further, the output control unit 108 may set, as the inputsignals of the other haptic output actuator, a waveform obtained byusing the BPF on the generated waveform. For example, the other hapticoutput actuator corresponding to the BPF may be an eccentric motor, andthe other haptic output actuator that sets, as the input signals, awaveform obtained by using the BPF may be an LRA. In this case, sincethe filters and haptic output actuators that are suitable for thefrequency distribution of the waveform serving as input signals areused, it becomes possible to increase the sense of reality or hapticvibrations.

2-2. Processing of Information Processing System

Next, processing of the information processing system in the presentembodiment will be described with reference to FIG. 5. FIG. 5 is asequence diagram conceptually showing the processing of the informationprocessing system according to the present embodiment.

First, the controller 200-1 detects a user's operation (step S302).Specifically, the operation detection unit 202 detects the user'soperation with respect to the controller 200-1, and generates theoperation information indicating the detected operation content.

Next, the controller 200-1 transmits information on the detectedoperation to the information processing apparatus 100-1 (step S304).Specifically, the communication unit 204 transmits the operationinformation, generated by the operation detection unit 202, to theinformation processing apparatus 100-1.

Then, the information processing apparatus 100-1 decides the movement ofan object from the received operation information (step S306).Specifically, the operation processing unit 104 performs movingprocessing of the object in the virtual space on the basis of theoperation information received by the communication unit 102.

Next, the information processing apparatus 100-1 generates a waveform onthe basis of the movement of the object (step S308). Specifically, whenone of the objects is moved relatively by the operation processing unit104 in a state in which the two objects are in contact with each otherin the virtual space, the waveform generation unit 106 generates thewaveform on the basis of the change of the acquisition position of theheight of a convex and a concave in the contact face between the twoobjects.

Next, the information processing apparatus 100-1 filters the generatedwaveform to generate the haptic output signals and sound output signals(step S310). Specifically, the output control unit 108 generates thehaptic output signals by using the LPF on the waveform generated by thewaveform generation unit 106, and generates the sound output signals byusing the HPF on the generated waveform.

Next, the information processing apparatus 100-1 transmits the soundoutput signals and haptic output signals to the controller 200-1 (stepS312). Specifically, the communication unit 102 transmits the generatedsound output signals and haptic output signals to the controller 200-1.

Then, the controller 200-1 performs the sound output on the basis of thereceived sound output signals (step S314). Specifically, the soundoutput unit 208 performs the sound output on the basis of the soundoutput signals received by the communication unit 204.

Next, the controller 200-1 performs the haptic output on the basis ofthe received haptic output signals (step S316). Specifically, the hapticoutput unit 206 performs the haptic output on the basis of the hapticoutput signals received by the communication unit 204.

In this manner, according to the first embodiment of the presentdisclosure, the information processing system generates the waveformaccording to the relative movement between the objects in the virtualspace, and performs the haptic output and sound output on the basis ofthe haptic output signals and sound output signals generated on thebasis of the frequency distribution of the generated waveform.Accordingly, since the waveform related to frequencies that are notoutputted by the haptic output actuator is outputted as sounds, itbecomes possible to relieve the user's feeling of strangeness withrespect to the output.

2-3. Modification Example

The first embodiment of the present disclosure has been described above.Note that the present embodiment is not limited to the above example. Amodification example of the present embodiment will be described below.

As a modification example of the present embodiment, the filteringintensity of filtering performed by the output control unit 108 may bechanged according to the speed of the relative movement between theobjects. Specifically, the operation processing unit 104 calculates thespeed of the relative movement between the objects for every certainperiod of time, and the output control unit 108 changes the filteringintensity of the LPF and the HPF according to the calculated speed ofthe objects. For example, the output control unit 108 can normalize thecalculated speed of the objects by performing division by a certainreference speed or the like, and give the normalized value to the LPFand the HPF. As the given value is larger, that is, as the speed of theobjects is higher, the LPF can make the filtering intensity higher andattenuate frequencies higher than the certain frequency more. As thegiven value is larger, the HPF can make the filtering intensity lowerand reduce attenuating rate of frequencies lower than the certainfrequency.

In this manner, according to the modification example of the presentembodiment, the filtering intensity of filtering performed by the outputcontrol unit 108 is changed according to the speed of the relativemovement between the objects. Accordingly, since an output that fits theheight of the frequency distribution of the generated waveform isemphasized, it becomes possible to promote the user to notice the changein frequencies and to increase the sense of reality for an outputcorresponding to a user's operation.

3. Second Embodiment of Present Disclosure (Example of Output Based onOperation Pressure)

Next, an information processing system according to the secondembodiment of the present disclosure will be described. In theinformation processing system according to the present embodiment, acontroller 200-2 detects an operation pressure, and the informationprocessing apparatus 100-2 changes the signal intensity of the generatedwaveform according to the detected operation pressure.

3-1. Configuration of Information Processing System

First, a configuration of the information processing system according tothe second embodiment of the present disclosure will be described withreference to FIG. 6. FIG. 6 is a block diagram showing a schematicfunctional configuration of the information processing system accordingto the present embodiment.

As shown in FIG. 6, the controller 200-2 includes, in addition to theoperation detection unit 202, the communication unit 204, the hapticoutput unit 206, and the sound output unit 208, a pressure detectionunit 220.

The pressure detection unit 220 detects the pressure related to a user'soperation with respect to the controller 200-2. For example, thepressure detection unit 220 can detect a pressing force related to theoperation on an operation unit (not shown), such as a touch panel, witha finger of the user or the like, the operation unit also being includedin the controller 200-2. For example, the pressure detection unit 220can be a pressure-sensitive sensor that detects the pressure from achange in an electrostatic capacitance or the like.

The operation detection unit 202 detects the position of the user'soperation with respect to the controller 200-2. Specifically, theoperation detection unit 202 detects the position of the user'soperation with respect to the above described operation unit, andgenerates the operation information indicating the detected operationposition and pressure information indicating the pressure detected bythe pressure detection unit 220, the pressure corresponding to theoperation position.

As shown in FIG. 6, further, the information processing apparatus 100-2includes, in addition to the communication unit 102, the operationprocessing unit 104, the waveform generation unit 106, and the outputcontrol unit 108, an amplification unit 120.

The amplification unit 120 decides the signal intensity of the waveformgenerated by the waveform generation unit 106 on the basis of thepressure information received by the communication unit 102.Specifically, when the waveform generation unit 106 generates thewaveform, the amplification unit 120 adjusts the signal intensity of thegenerated waveform according to the strength of the pressure indicatedby the pressure information. For example, processing of theamplification unit 120 will be described in detail with reference toFIG. 7. FIG. 7 shows an example of adjusting processing of the signalintensity related to the waveform generated from the operationinformation of the information processing apparatus 100-2 according tothe present embodiment.

First, the operation detection unit 202 detects the position of theuser's operation with respect to the controller 200-2, and the pressuredetection unit 220 detects the pressure at the position. For example, asshown in the left of FIG. 7, when the user 1 touches the operation unitsuch as a touch panel of the controller 200-2 with her/his finger, theoperation detection unit 202 detects the position of the finger of theuser 1 on the touch panel, and the pressure detection unit 220 detectsthe pressure at the position of the finger. Further, when the user 1moves the finger from the left to the right in the figure in thehorizontal direction, for example, in a state in which the finger is incontact with the touch panel, the operation detection unit 202 detectsthe displacement of the position of the finger, and the pressuredetection unit 220 detects the pressure in each position of the finger.Then, the operation information indicating the detected position of thefinger and pressure information indicating the pressure at the positionare transmitted to the information processing apparatus 100-2 by thecommunication unit 204.

Next, the operation processing unit 104 associates the operationposition indicated by the operation information received by thecommunication unit 102 with the position in the virtual space, moves theposition of an object on the basis of the displacement of the operationposition, and changes the shape of the object in a contact partaccording to the strength of the pressure indicated by the pressureinformation received by the communication unit 102.

Next, the waveform generation unit 106 generates the waveform accordingto the relative movement between the objects. This processing issubstantially the same as the processing in the first embodiment, andtherefore a detailed description thereof is omitted here.

When the operation processing unit 104 makes the two objects be incontact with each other or moves an object relatively in a state inwhich the object is in contact with the other object, the amplificationunit 120 acquires the pressure information in a contact face between thetwo objects. For example, the operation information received by thecommunication unit 102 can be stored in a storage unit also included inthe information processing apparatus 100-2, and the amplification unit120 can acquire, from the storage unit, the pressure information at theoperation potion indicated by the operation information corresponding tothe contact face between the objects.

Next, when the waveform generation unit 106 generates a waveform, theamplification unit 120 changes the signal intensity of the generatedwaveform according to the strength of the pressure indicated by theacquired pressure information. For example, when the waveform shown inthe middle of FIG. 7 is generated by the waveform generation unit 106,the amplification unit 120 increases the signal intensity as shown inthe right of FIG. 7 according to the strength of the pressure indicatedby the acquired pressure information. For example, as the pressureindicated by the acquired pressure information is stronger, theamplification unit 120 can increase the signal intensity of thewaveform.

In this manner, the amplification unit 120 decides the signal intensityof the waveform generated by the waveform generation unit 106 accordingto the user's operation pressure, that is, the pressure on the contactface between the objects. Accordingly, since the output is changedaccording to the detected pressure, it becomes possible to make the userto sense a more realistic feedback with respect to the user's operation.

Here, referring to FIG. 6 again, the configuration of the informationprocessing apparatus 100-2 will be described again. In a case in whichthe signal intensity of frequencies included in the frequencydistribution of the waveform is higher than a certain threshold, theoutput control unit 108 causes an output. Specifically, in a case inwhich the signal intensity of the waveform obtained by using the LPF orthe HPF is higher than the certain threshold, the output control unit108 sets the obtained waveform as output signals. For example, controlprocessing of output signals by using the signal intensity will bedescribed in detail with reference to FIG. 8. FIG. 8 shows anotherexample of a change in the frequency distribution of the waveformgenerated in the information processing apparatus 100-2 according to thepresent embodiment.

First, when the two objects are made in contact with each other andrelatively moved, the waveform generation unit 106 generates a waveformon the basis of the height of a convex and a concave of an object. Forexample, frequencies of the generated waveform can be distributed asshown in an upper part of FIG. 8.

Next, the output control unit 108 filters the generated waveform byusing the LPF or the HPF. For example, the frequency distribution of thewaveform obtained by using the LPF can be the distribution offrequencies represented as white bars in the upper part of FIG. 8, andthe frequency distribution of the waveform obtained by using the HPF canbe the distribution of frequencies represented as hatched bars in theupper part of FIG. 8.

Next, the output control unit 108 sets, as output signals, a waveform inwhich the signal intensity of the waveform obtained by filtering ishigher than the certain threshold. For example, as shown in the upperpart of FIG. 8, since the waveform obtained by using the LPF includesfrequencies having signal intensity higher than the certain threshold,such as the broken line in the upper part of FIG. 8, the output controlunit 108 can set the obtained waveform as the haptic output signals. Incontrast, since the waveform obtained by using the HPF includesfrequencies having signal intensity lower than the broken line in theupper part of FIG. 8, the output control unit 108 does not set theobtained waveform as sound output signals.

In contrast, when only the user's operation pressure is increased andthe same movement between the objects is performed at the same speed,the waveform generated by the waveform generation unit 106 does not havedifferent frequency distribution, but the signal intensity is increasedin the entire frequencies that are distributed. For example, as shown ina lower part of FIG. 8, compared with the frequency distribution of theupper part in which the pressure is relatively low, the height offrequencies is not changed and the signal intensity can be increased.

Then, since the signal intensity of frequencies of the waveform obtainedby using the HPF shown in the lower part of FIG. 8 is higher than thebroken line shown in the lower part of FIG. 8, the output control unit108 can set the obtained waveform as sound output signals.

In this manner, in a case in which the signal intensity of frequenciesincluded in the frequency distribution of the generated waveform ishigher than the certain threshold, the output control unit 108 causes anoutput. Accordingly, since the presence and absence of an output isemphasized, it becomes possible to promote the user to notice a changein the signal intensity and to further increase the sense of reality ofthe output corresponding to the user's operation.

3-2. Processing of Information Processing System

Next, processing of the information processing system in the presentembodiment will be described with reference to FIG. 9. FIG. 9 is asequence diagram conceptually showing the processing of the informationprocessing system according to the present embodiment. Note that adetailed description of processing that is substantially the same asthat of the information processing system in the first embodiment willbe omitted.

First, the controller 200-2 detects a user's operation and the pressurerelated to the operation (step S402). Specifically, the pressuredetection unit 220 detects the pressure related to the user's operation,and the operation detection unit 202 detects the position of the user'soperation.

Next, the controller 200-2 transmits information on the detectedoperation and pressure to the information processing apparatus 100-2(step S404). Specifically, the operation detection unit 202 generatesoperation information indicating the detected operation position and thepressure detected by the pressure detection unit 220, and thecommunication unit 204 transmits the generated operation information tothe information processing apparatus 100-2.

Next, the information processing apparatus 100-2 decides the movement ofan object from the received operation information (step S406).Specifically, the operation processing unit 104 moves the position ofthe object on the basis of the operation information received by thecommunication unit 102, and changes the shape of the object in a contactpart according to the pressure indicated by the operation information.

Next, the information processing apparatus 100-2 generates a waveform onthe basis of the decided movement of the object (step S408).Specifically, the information processing apparatus 100-2 performssubstantially the same processing as in the step S308 in the firstembodiment.

Next, the information processing apparatus 100-2 adjusts the signalintensity of the generated waveform on the basis of the receivedinformation on the pressure (step S410). Specifically, when the waveformgeneration unit 106 generates the waveform, the amplification unit 120changes the signal intensity of the generated waveform according to thestrength of the pressure indicated by the received operationinformation.

Next, the information processing apparatus 100-2 filters the generatedwaveform to generate the haptic output signals and sound output signals(step S412). Specifically, in a case in which the waveform obtained byfiltering the waveform in which the signal intensity is adjustedincludes a frequency that is higher than the certain threshold, theoutput control unit 108 sets the waveform as output signals.

Next, the information processing apparatus 100-2 transmits the soundoutput signals and haptic output signals to the controller 200-2 (stepS414). Specifically, the information processing apparatus 100-2 performssubstantially the same processing as in the step S312 in the firstembodiment.

Then, the controller 200-2 performs the sound output on the basis of thereceived sound output signals (step S416), and performs the hapticoutput on the basis of the received haptic output signals (step S418).

In this manner, according to the second embodiment of the presentdisclosure, the information processing system detects the pressurerelated to the user's operation and changes the signal intensity of thegenerated waveform according to the detected pressure. Accordingly,since the output is changed according to the detected pressure, itbecomes possible to make the user sense a more realistic feedback withrespect to the user's operation.

4. Hardware Configuration

Embodiments of the present disclosure have been described above. Theprocessing of the information processing apparatus 100 described aboveis implemented by cooperation between software and hardware of theinformation processing apparatus 100 described below.

FIG. 10 is an explanatory diagram showing a hardware configuration ofthe information processing apparatus 100 according to an embodiment ofthe present disclosure. As illustrated in FIG. 10, the informationprocessing apparatus 100 includes a central processing unit (CPU) 132,read only memory (ROM) 134, random access memory (RAM) 136, a bridge138, a bus 140, an interface 142, an input device 144, an output device146, a storage device 148, a drive 150, a connection port 152, and acommunication device 154.

The CPU 132 functions as an arithmetic processing device and a controldevice and cooperates with various programs to thereby realize theoperations of the operation processing unit 104, the waveform generationunit 106, the output control unit 108, and the amplification unit 120 inthe information processing apparatus 100. The CPU 132 may be amicroprocessor. The ROM 134 stores programs, operation parameters, orthe like to be used by the CPU 132. The RAM 136 temporarily storesprograms for use in the execution of the CPU 132, parameters whichchange as appropriate in the execution, and the like. By the ROM 134 andthe RAM 136, a part of the storage unit in the information processingapparatus 100 is realized. The CPU 132, the ROM 134, and the RAM 136 areconnected to each other through an internal bus constituted by a CPU busand the like.

The input device 144 is constituted by, for example, an input unit whichallows the user to input information, such as a mouse, a keyboard, atouch panel, a button, a microphone, a switch, and a lever, an inputcontrol circuit which generates an input signal based on an input by theuser, and then outputs the signal to the CPU 132, and the like. The userof the information processing apparatus 100 can operate the input device144 to input various data to the information processing apparatus 100 orinstruct the information processing apparatus 100 to perform aprocessing operation.

The output device 146 performs an output operation to a device, such asa liquid crystal display (LCD), an organic light emitting diode (OLED),and a lamp. Furthermore, the output device 146 may include a loudspeakerand a headphone for outputting sounds.

The storage device 148 is a device for data storage. The storage device148 may include a storage medium, a recording device for recording datain the storage medium, a reading device for reading the data from thestorage medium, and a deleting device for deleting data recorded in thestorage medium. The storage device 148 stores a program executed by theCPU 132 and various types of data.

The drive 150 is a reader-writer for a storage medium and is built in orexternally attached to the information processing apparatus 100. Thedrive 150 reads out information recorded on a removable storage mediumthat is attached thereto, such as a magnetic disk, an optical disk, amagneto-optical disk, or a semiconductor memory, and then outputs theinformation to the RAM 134. The drive 150 is also able to writeinformation to a removable storage medium.

The connection port 152 is a bus for connection with an informationprocessing apparatus or peripheral devices provided outside theinformation processing apparatus 100. The connection port 152 may beuniversal serial bus (USB).

The communication device 154 is, as an example of the communication unit102 of the information processing apparatus 100, a communicationinterface constituted by a communication device for connection with anetwork. The communication device 154 may be an infrared communicationcompatible device, a wireless local area network (LAN) compatiblecommunication device, a long-term evolution (LTE) compatiblecommunication device, or a wired communication device that performscommunication through wire.

5. Conclusion

As described above, according to the first embodiment of the presentdisclosure, since the waveform related to frequencies that are notoutputted by the haptic output actuator is outputted as sounds, itbecomes possible to relieve the user's feeling of strangeness withrespect to the output. According to the second embodiment of the presentdisclosure, since the output is changed according to the detectedpressure, it becomes possible to make the user sense a more realisticfeedback with respect to the user's operation.

Hereinabove, although the preferred embodiments of the presentdisclosure have been described in detail with reference to theaccompanying drawings, the technical scope of the present disclosure isnot limited thereto. It should be understood by those skilled in the artthat various modifications, combinations, sub-combinations andalterations may occur depending on design requirements and other factorsinsofar as they are within the scope of the appended claims or theequivalents thereof.

For example, the above embodiments have shown the examples in which theheight of a convex and a concave of the object is transformed into thesignal intensity and the change of the acquisition position of theheight of a convex and a concave is represented as the waveform;however, the present technology is not limited to the above examples.For example, the waveform generation unit 106 may generate the waveformon the basis of the waveform of certain frequency distribution.Specifically, when the relative movement occurs in a state in which theobjects are in contact with each other, the waveform generation unit 106acquires the waveform of the certain frequency distribution stored inthe storage unit, and varies frequencies in the acquired waveformaccording to the speed of the relative movement between the objects. Forexample, when the speed of the relative movement between the objects isincreased, the frequencies of the waveform are changed in a manner thatthe frequency distribution of the acquired waveform is shifted to be ina higher region. In this case, since the processing of generating thewaveform is simplified, it becomes possible to reduce the processingload and to increase the processing speed. Note that the waveform of thecertain frequency distribution may differ according to the material ofthe object or the like.

Further, the controller 200 may further include a sound output controlunit, and the sound output control unit may cause the sound output unit208 to perform the sound output according to the start of the operationof the haptic output actuator. Specifically, when the communication unit204 receives the sound output signals, the sound output control unitholds the sound output signals temporarily, and after a certain periodof time elapses, the sound output control unit provides the sound outputsignals to the sound output. Note that the certain period of time can bestored in the storage unit that is also included in the controller 200.In this case, by the sound output being performed according to a delayof the operation of the haptic output actuator, it becomes possible toprevent the user from experiencing the feeling of strangeness.

It should be noted that in the present disclosure, the operationsdescribed in the sequence diagram may be performed not only in timeseries in the described order but in parallel or individually. Further,it should be understood that the operations performed in time series maybe performed in a different order from the described order.

In addition, the effects described in the present specification aremerely illustrative and demonstrative, and not limitative. In otherwords, the technology according to the present disclosure can exhibitother effects that are evident to those skilled in the art along with orinstead of the effects based on the present specification.

Additionally, the present technology may also be configured as below.

(1) An information processing apparatus including:

a waveform generation unit configured to generate a waveform accordingto a relative movement between two objects in a virtual space; and

an output control unit configured to control a haptic output and a soundoutput on the basis of frequency distribution of the waveform generatedby the waveform generation unit.

(2) The information processing apparatus according to (1),

wherein the output control unit controls the sound output in a case inwhich the frequency distribution of the waveform includes a frequencyhigher than a certain frequency, and controls the haptic output in acase in which the frequency distribution of the waveform includes afrequency lower than the certain frequency.

(3) The information processing apparatus according to (2),

wherein the sound output is an output of a waveform related to thefrequency higher than the certain frequency in the waveform, and

wherein the haptic output is an output of a waveform related to thefrequency lower than the certain frequency in the waveform.

(4) The information processing apparatus according to any one of (1) to(3),

wherein the waveform generation unit generates a waveform having afrequency distribution according to a speed of the relative movementbetween the objects.

(5) The information processing apparatus according to any one of (1) to(4),

wherein the output control unit causes an output in a case in which asignal intensity of a frequency included in the frequency distributionof the waveform is higher than a certain threshold.

(6) The information processing apparatus according to (5),

wherein the waveform generation unit decides the signal intensityaccording to a pressure between the objects.

(7) The information processing apparatus according to (2) or (3),

wherein the output control unit sets the waveform as input signals ofthe haptic output and the sound output.

(8) The information processing apparatus according to (7),

wherein the output control unit sets, as the input signal of the soundoutput, a waveform obtained by filtering the frequency lower than thecertain frequency from the frequency distribution of the waveform.

(9) The information processing apparatus according to (7) or (8),

wherein the output control unit sets, as the input signal of the hapticoutput, a waveform obtained by filtering the frequency higher than thecertain frequency from the frequency distribution of the waveform.

(10) The information processing apparatus according to (8) or (9),

wherein in the filtering, a filtering intensity is varied according to aspeed of the relative movement between the objects.

(11) The information processing apparatus according to any one of (1) to(10),

wherein at least one of the objects is moved by a user's operation.

(12) The information processing apparatus according to any one of (1) to(11),

wherein the waveform generation unit generates a waveform according toroughness information of a contact face between the objects.

(13) An information processing system including:

a waveform generation unit configured to generate a waveform accordingto a relative movement between two objects in a virtual space; and

an output control unit configured to control a haptic output and a soundoutput on the basis of frequency distribution of the waveform generatedby the waveform generation unit.

(14) An information processing method including:

generating a waveform according to a relative movement between twoobjects in a virtual space; and

controlling a haptic output and a sound output on the basis of frequencydistribution of the generated waveform.

(15) A program for causing a computer to execute:

a waveform generation function of generating a waveform according torelative movement between two objects in a virtual space; and

an output control function of controlling a haptic output and a soundoutput on the basis of frequency distribution of the waveform generatedby the waveform generation function.

What is claimed is:
 1. An information processing apparatus, comprising:at least one processor configured to: control reception of operationinformation from a device connected to the information processingapparatus, wherein the operation information indicates a user operationof the device; determine a relative movement between two objects in avirtual space based on the user operation of the device; controlgeneration of a first waveform based on the relative movement betweenthe two objects in the virtual space, wherein frequency distribution ofthe generated first waveform is based on a speed of the relativemovement between the two objects; control a haptic output and a soundoutput of the device based on the frequency distribution of thegenerated first waveform; control the sound output based on thefrequency distribution that includes a first frequency higher than athreshold frequency; and control the haptic output based on thefrequency distribution that includes a second frequency lower than thethreshold frequency.
 2. The information processing apparatus accordingto claim 1, wherein the sound output is an output of a second waveformrelated to the first frequency in the first waveform, wherein the firstfrequency is higher than the threshold frequency, and wherein the hapticoutput is an output of a third waveform related to the second frequencyin the first waveform, wherein the second frequency is lower than thethreshold frequency.
 3. The information processing apparatus accordingto claim 1, wherein the at least one processor is further configured tocontrol the haptic output and the sound output based on a signalintensity of a third frequency included in the frequency distribution ofthe first waveform, wherein the signal intensity is higher than athreshold.
 4. The information processing apparatus according to claim 3,wherein the at least one processor is further configured to determinethe signal intensity based on a pressure between the two objects.
 5. Theinformation processing apparatus according to claim 1, wherein the atleast one processor is further configured to set the first waveform asan input signal of the haptic output and the sound output.
 6. Theinformation processing apparatus according to claim 1, wherein the atleast one processor is further configured to set, as an input signal ofthe sound output, a second waveform obtained based on filtration of thesecond frequency from the frequency distribution of the first waveform,wherein the second frequency is lower than the threshold frequency. 7.The information processing apparatus according to claim 1, wherein theat least one processor is further configured to set, as an input signalof the haptic output, a third waveform obtained based on filtration ofthe first frequency from the frequency distribution of the firstwaveform, wherein the first frequency is higher than the thresholdfrequency.
 8. The information processing apparatus according to claim 6,wherein the at least one processor is further configured to: controlfiltration of the generated first waveform; and vary a filteringintensity of the filtration based on the speed of the relative movementbetween the two objects.
 9. The information processing apparatusaccording to claim 1, wherein at least one object of the two objects ismoved based on the user operation.
 10. The information processingapparatus according to claim 1, wherein the at least one processor isfurther configured to control the generation of the first waveform basedon roughness information of a contact face between the two objects. 11.An information processing system, comprising: an information processingapparatus that comprises at least one processor, wherein the at leastone processor is configured to: control generation of a waveform basedon a relative movement between two objects in a virtual space, whereinfrequency distribution of the generated waveform is based on a speed ofthe relative movement between the two objects; control a haptic outputand a sound output based on the frequency distribution of the generatedwaveform; control the sound output based on the frequency distributionthat includes a first frequency higher than a threshold frequency; andcontrol the haptic output based on the frequency distribution thatincludes a second frequency lower than the threshold frequency.
 12. Aninformation processing method, comprising: controlling reception ofoperation information from a device connected to an informationprocessing apparatus, wherein the operation information indicates a useroperation of the device; determining a relative movement between twoobjects in a virtual space based on the user operation of the device;generating a waveform based on the relative movement between the twoobjects in the virtual space, wherein frequency distribution of thegenerated waveform is based on a speed of the relative movement betweenthe two objects; controlling a haptic output and a sound output of thedevice based on the frequency distribution of the generated waveform;controlling the sound output based on the frequency distribution thatincludes a first frequency higher than a threshold frequency; andcontrolling the haptic output based on the frequency distribution thatincludes a second frequency lower than the threshold frequency.
 13. Anon-transitory computer-readable storage medium having stored thereoncomputer-executable instructions for causing a computer to performoperations, the operations comprising: controlling reception ofoperation information from a device connected to an informationprocessing apparatus, wherein the operation information indicates a useroperation of the device; determining a relative movement between twoobjects in a virtual space based on the user operation of the device;generating a waveform based on the relative movement between the twoobjects in the virtual space, wherein frequency distribution of thegenerated waveform is based on a speed of the relative movement betweenthe two objects; controlling a haptic output and a sound output of thedevice based on the frequency distribution of the generated waveform;controlling the sound output based on the frequency distribution thatincludes a first frequency higher than a threshold frequency; andcontrolling the haptic output based on the frequency distribution thatincludes a second frequency lower than the threshold frequency.